The challenges of evaluating and training individuals has led to many developments over the years, and computer automated testing and training has become the norm in many fields. The field of academics has proven to be one fertile ground for development of computer automated testing and training, and training for standardized examinations with practice tests and questions is only one example application area. Professional training has proven to be another fertile ground for development of computer automated training, and such training proves particularly useful in professions that primarily require computer interfacing skills.
Skills evaluation is related to training because assessing an individual's skills before training the individual assists a training tool in focusing on an individual's training needs. In fact, one past solution to the challenge of training individuals has involved testing an individual's ability to perform tasks with a wide range of difficulty level, and then training them on the tasks they failed to perform correctly. The inefficiencies inherent in this approach were recognized nearly half a century ago. However, it was not until the advent of high-speed computers in the nineteen-seventies that Frederic Lord and others could construct a computer adaptive testing (CAT) alternative to mass, full-item testing. For more on this point, refer to Howard Wainer, Computerized Adaptive Testing: A Primer (Hillsdale, N.J. 1990), pp. 8-11.
With CAT, task items are ranked according to difficulty level and a computer selects task items for a user during an exam based on the user's performance during the exam. For example, if a user fails to perform a task of a certain difficulty level, then the computer selects a new task for the user with somewhat lower level of difficulty. Also, if the user succeeds in performing a task of a certain difficulty level, then the computer selects a new task for the user of a somewhat higher level of difficulty. This function is performed recursively until the computer determines a user's ability based on overall performance during the test, and the CAT system has advantages over previous skills assessment techniques.
A distinct advantage of the CAT system is the ability to evaluate a user's skill level with a fewer number of tasks, thereby saving significant amounts of time and effort. For example, performance of fifteen task items is sufficient with a CAT system compared with a requirement for performance of seventy-five task items with previous evaluation systems. Thus, CAT systems benefit from the ability to minimize the number of items required to measure accurately a person's ability. This advantage, however, has a tradeoff when combined with previous training techniques, because training users only on missed items following a CAT exam fails to provide adequate training.
Unsuitability of combining previous training procedures with a CAT system relates to the advantages of the CAT system. For example, this unsuitability relates to the ability of the CAT system to assess an individual's skill level with a fewer number of tasks. Unlike previous testing procedures, the CAT system generally results in a smaller number of missed items during an exam regardless of a user's skill level. Also, the unsuitability relates to the ability of the CAT system to rapidly converge on a person's ability level and thereby eliminate the need to test a user on tasks far below, or far above, their ability. As a result of the testing technique granting this ability, difficulty levels of missed items usually congregate at or near a particular level. Thus, training users only on missed items generally results in a small amount of training at or near a single level of difficulty.
Given the development and availability of the CAT skills assessment technique, the need arises for an adaptive training system and method for use with the CAT tool. Existing training methods, such as training users only on tasks they failed to perform correctly during the CAT test, fails to take into account the following: (1) The CAT test is not converging simply on items previously missed, but rather on items close to a trained person's higher measured ability level; (2) if training after one CAT test is effective, it will have a “Heisenberg” effect on the path taken by the next CAT, such that with his/her measured ability improved, the trained person will now experience a more difficult CAT test; and (3) it is only when the training has been ineffective that the second CAT will more or less duplicate the path of the first CAT. Thus, training users only on tasks they failed to perform correctly on a CAT test merely trains user's on a handful of tasks they are unlikely to encounter on a subsequent CAT exam. Such a training method also fails to focus on the goal of raising an individual's overall skill level. Therefore, the need remains for a solution to the challenge of training individuals based on an adaptive skills evaluation technique that focuses on raising overall skill level in a desired skill set. The present invention provides such a solution.